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There is a widespread view in the research and 
policy communities that the quality of mathemat- 
ics and science instruction offered to students in 
the United States is low (Business-Higher Educa- 
tion Forum, 2005; Gonzales et ah, 2004; National 
Commission on Mathematics and Science Teach- 
ing for the 2U‘ Century, 2000; National Research 
Council, 2000, 2001, 2005; National Science 
Board, 2004; Weiss, Pasley, Smith, Banilower, & 
Heck, 2003). The widely discussed results of the 
Trends in International Mathematics and Science 
Study (TIMSS) suggested that the reasons for 
American students’ poor performance in mathe- 
matics and science are complex, but at least part- 
ly due to weaknesses in the knowledge and skills 
of those teaching the subjects (Schmidt et ah, 
2001). In order to enhance teaching in these con- 
tent areas, states and school districts need to act 
on what research has discovered about profes- 
sional development (PD) and instructional 
improvement. The purpose of this brief is to share 
results from a major PD effort that extend our 
understanding of what is entailed in creating 
improvements at scale. 

The Consensus View 

A review of research over the last decade 
shows that typical PD, limited to a few in-service 
days a year, is ineffective in enhancing teaching 
(Corcoran, 1995). Elmore (2002) urges that in 
order to produce sustained gains in student learn- 
ing, adequate opportunities for teachers to 
enhance both their pedagogical skill and content 
knowledge must be provided. Yet in spite of such 
recommendations and the pressures emanating 
from higher-stakes accountability systems, most 
PD opportunities remain fragmented, poorly 
aligned with curricula, and inadequate to meet 
teachers’ needs — particularly acute in mathemat- 



ics and science — for deeper knowledge of subject 
matter and understanding of pedagogy (Cohen & 
Hill, 2001; Corcoran & McDiarmid, 2000). Often 
the content of existing PD programs is unconnect- 
ed to teachers’ daily work, and little follow-up 
support is offered. 

In recent years, researchers have begun to 
examine what works in PD. Supovitz, Mayer, and 
Kahle (2000) studied the effects of intensive, stan- 
dards-based PD on science teachers in Ohio. They 
found that teachers became more positive about 
instructional reforms and more likely to use 
inquiry-centered pedagogy as a result of partici- 
pating in intensive, standards-based PD. Cohen 
and Hill (2001) studied mathematics teachers par- 
ticipating in intensive curriculum-based PD 
offered by California in the 1990s. They found 
that participants were much more likely to make 
changes in practice than teachers receiving more 
general PD; these changes were associated with 
student performance gains. They concluded that 
providing teachers with extended, content-specif- 
ic opportunities combined with follow-up support 
produced results. These findings are consistent 
with those of other studies, including a secondary 
analysis of the National Science Foundation 
(NSF) project that is the subject of this brief 
(Supovitz & Turner, 2000). The general conclu- 
sion is that extended PD opportunities aligned 
with curricular content and accompanied by on- 
site follow-up support can produce significant 
changes in classroom practice and benefits for 
students. 

On the basis of such studies, researchers and 
PD designers have reached broad agreement on 
key features of effective PD programs that lead to 
changes in teaching practices. (Elmore, 2002; 
Loucks-Horsley, Love, Stiles, Mundry, & Hew- 
son., 2003; Wilson & Berne, 1999). According to 
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this consensus view, high-quality PD programs 
are grounded in research and clinical knowledge 
of teaching and learning. They are aligned with a 
school’s curriculum and assessments and focused 
on student learning in that setting. They facilitate 
teachers’ collaboration both within and across 
schools, they use existing teacher expertise to plan 
activities and cultivate leaders, and they include 
mechanisms for garnering principal support. 
High-quality PD programs both model and explic- 
itly discuss methods of good practice (such as 
inquiry-based methods in science) and provide 
teachers with active learning opportunities. These 
programs aim to build teachers’ content knowl- 
edge and pedagogical skills. Finally, they are 
intensive, sustained over time to allow for inte- 
gration of new knowledge into practice, and 
include follow-up support. While PD designed 
according to these principles does not guarantee 
that participating teachers will use what they learn 
to change their practice, it increases the likelihood 
that they will. 

The Local Systemic Change Initiative 

One major effort to provide teachers with such 
sustained, content-specific PD was instituted with 
NSF support in 1995. NSF initiated its Local Sys- 
temic Change through Teacher Enhancement 
(LSC) program in 1995 in order to improve 
instruction in science, mathematics, and technolo- 
gy through teacher PD. NSF funded the first 
cohort of LSC projects in 1995, and a new cohort 
of projects was added each year. By 2002, a total 
of 88 projects across the country had received 
LSC funding, typically for a period of five years 
each, with some projects using the first year for 
planning. By 2005, the LSC initiative had reached 

70.000 elementary and secondary teachers (pre- 
dominantly K-8) and two million students in 

4.000 schools. The schools and students served by 
the LSC initiative included a high proportion of 
historically underrepresented groups. Nearly half 
of the schools targeted by the LSC were in urban 
areas, a quarter in suburban areas, and the rest 
about equally split between rural areas and towns 
or small cities. Across all schools targeted by the 
LSCs, just over half of the students were from 
minority groups. 

The initiative aimed to involve all of a juris- 
diction’s teachers of the targeted subjects in par- 
ticular grade bands in project-designated PD. 
Each targeted teacher was to participate in a min- 
imum of 130 hours of PD over the course of the 
project. PD activities aimed to prepare teachers to 
implement high-quality mathematics and science 
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materials in their classes and to use inquiry-based 
practices that the materials supported. The chief 
aim of the LSC program was to encourage large- 
scale reform in teaching practice by providing 
high-quality PD to all teachers and by supporting 
institutionalization of PD systems featuring 
enhanced practices and use of high-quality mate- 
rials. 

The PD provided by LSC projects proceeded 
from a clear theory of action in accordance with 
the research described above. The theory held that 
providing teachers with opportunities to deepen 
their content and pedagogical knowledge in the 
context of high-quality instructional materials 
would result in better-prepared teachers. The the- 
ory also predicted that with ongoing support, 
these teachers would be more inclined to change 
their instruction in ways advocated by national 
standards and would have more capacity to do so. 
Improved instruction would in turn lead to higher 
student achievement. This theory of action 
entailed common principles of high-quality PD 
opportunities that could improve teaching, of 
which three were cmcial: establishing a support- 
ive PD culture; providing PD experiences ground- 
ed in a combination of content, pedagogy, and 
materials; and using expert providers. 

To realize the first principle, the initiative pro- 
moted efforts beyond the PD activities themselves 
to build a supportive environment for improving 
mathematics and science instruction. Eirst, LSCs 
were charged with building a shared vision of 
mathematics and science education, designing a 
strategic plan for engaging teachers in PD, and 
promoting active partnerships and commitments 
among an array of stakeholders. LSC projects 
were also expected to align policy and practice 
within targeted districts and to engage in a range 
of activities to support reform. These included 
aligning mathematics and science curriculum 
frameworks, teacher evaluation systems, and stu- 
dent assessments with the LSC reforms. 

To realize the second principle, designs for PD 
in individual projects typically included summer 
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institutes of a week or longer as well as school- 
year follow-up support, e.g., study groups, class- 
room demonstrations, and coaching. Whether 
centralized or school-based, the LSC designs var- 
ied widely according to local needs. For example, 
some projects provided a great deal of PD prior to 
the teachers’ implementing the instructional mate- 
rials for the first time, while others delayed PD 
until after teachers had used the materials in their 
classrooms. Some LSCs used a cohort approach, 
for example, targeting one third of the teachers at 
a time, while others targeted all teachers each 
year. Regardless of the specific design, most pro- 
jects estimated that it would take a teacher three or 
more years to reach the 130-hour target. 

Instruction in content tended to be at the level 
at which students were expected to learn. Nearly 
half of PD time was devoted to engaging teachers 
in mathematics or science inquiry, with teachers 
working through problems as students might, in 
groups facilitated by project PD providers. Usual- 
ly, teachers experienced about half of their PD 
hours in a large group, while about 40% took 
place in groups of under 15 people, and 10% in 
individual work, e.g., with coaches. Projects tried 
to weave together PD in content, materials, and 
pedagogy; time spent in these three areas was fair- 
ly balanced. According to evaluators, LSC 
designs were particularly effective when they 
guided teachers through a sequence of PD activi- 
ties from materials-based workshops to more 
advanced activities emphasizing students’ con- 
ceptual understanding and other pedagogical 
issues. Unlike typical PD, LSC work endeavored 
to create a year-round support structure for teach- 
ers by offering coaching and ongoing learning 
communities. However, for reasons described 
below, projects did not usually succeed in creating 
such a structure. 

LSC projects used a diverse group of PD 
providers to enact the third principle. Typically, 
projects tapped university and industry scientists 
and mathematicians to address participants’ con- 
tent needs in particular. University science and 
mathematics education faculty and others from 
outside school systems (such as museum person- 
nel) delivered PD in content, pedagogy, and mate- 
rials. Teacher leaders from within systems (both 
teachers on special assignment and regular class- 
room teachers) also played a significant role in 
PD delivery. The leaders served as school-level 
resources for follow-up PD and as liaisons with 
project staff. They collaborated on district PD 
design and problem solving. Leaders on special 



assignment had more responsibility for planning 
and implementing PD and for providing coaching. 
District mathematics and science supervisors also 
led activities. Commonly, university faculty and 
teacher leaders worked together in teams, with 
teacher leaders focusing on pedagogy and materi- 
als, and university experts focusing on conceptual 
understanding. 

In recmiting PD providers, projects sought 
individuals whose vision of mathematics and sci- 
ence instmction already matched that promoted 
by the LSC. At the same time, these providers rep- 
resented a range of experience in content, teach- 
ing, and reform leadership. LSCs attempted to 
monitor providers, help them define their roles, 
and build their knowledge and skills. Regular staff 
meetings were a primary mechanism for doing so; 
one experienced project developed an active PD 
system for supporting providers. Projects that had 
worked for a number of years with PD 
providers — through the LSC and previously fund- 
ed projects — often exhibited more highly devel- 
oped strategies for provider support, demonstrat- 
ing a key lesson learned from the initiative: that 
time matters. 

How well did the experience of the LSC pro- 
jects align with NSF’s theory of action? Horizon 
Research, Inc., worked with project evaluators in 
a cross-site evaluation focused on the quality and 
extent of PD activities and their impact on teach- 
ers’ attitudes and classroom practices in mathe- 
matics and science. Also addressed were ques- 
tions about the degree of district and school sup- 
port for the initiatives and about the extent of 
institutionalization of high-quality PD. Data gath- 
ered by evaluators of each project included obser- 
vations of PD activities and classroom practices, 
teacher and principal questionnaires, interviews 
with teachers and principal investigators (Pis), 
and project ratings and project strategies ques- 
tionnaires completed by Pis and LSC evaluators. 
Case studies and cross-site analyses were con- 
ducted as well. Banilower, Boyd, Pasley, and 
Weiss (2005) describe these measures and Hori- 
zon’s data analysis methods. 

This issue of CPRE Policy Briefs summarizes 
findings from the Horizon Research, Inc. evalua- 
tion. Broadly, three key insights emerge from the 
evaluation of the LSC effort. First, large-scale 
reform to enhance teaching should be carefully 
and specifically designed. Second, given such a 
design, system-wide PD can have significant 
impacts on instmction. Third, support across the 
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system facilitates effective implementation of 
such PD. The brief examines these insights and 
also underscores the key challenges and successes 
of the LSC initiative. It shows what the design and 
implementation of this ambitious attempt to apply 
the consensus view of effective PD to mathemat- 
ics and science accomplished. 

Design 

Large-scale PD reform must be carefully 
designed with particular features. While LSC 
designs were complex and varied considerably 
across projects, they contained common elements 
relevant to their impact. 

Goals 

Crucially, the implementation of LSC designs 
suggests that it is critical to set high and specific 
goals for the reform. NSF succeeded in that regard 
insofar as it was expected that the PD in each LSC 
would extend to all targeted teachers, be sustained 
over time, focus on content knowledge and use of 
research-based materials, be delivered in large 
measure by teacher leaders, and result in changes 
in practice. As will be shown, not all of those 
goals were attained perfectly, but the extent of 
attainment was impressive. These findings 
demonstrate that aiming high makes sense. Also, 
it helps to be prescriptive about goals while being 
flexible about implementation — this strategy 
proved to generate considerable creativity within 
individual LSCs. 

At the same time, LSC project leaders found 
that the multitude of goals made project design 
difficult. For example, while they wanted a deep 
focus on developing teacher knowledge, they also 
wanted broad exposure to student instmctional 
materials. Project staff were not always sure how 
to prioritize those goals. Moreover, designers 
wanted to counter the prevalent reform practice of 
encouraging teachers to develop/adapt instruc- 
tional materials, but they did not make this aim 
salient, and observers noted that teacher adapta- 
tions often led to decreased opportunities for stu- 
dent learning. Overall, the LSC experience indi- 
cated that it is critical to begin an initiative by 
making primary — and achievable — goals clear to 
all involved. 

Prior Knowledge 

An important feature of the design of the 
LSCs was that they built on prior knowledge of 
what works in PD for science and mathematics 
teachers. The LSCs used an inquiry-based 



approach to instruction and materials designed to 
support the approach, which has been demonstrat- 
ed by NSF studies and other research to benefit 
teacher knowledge and practice (for example, 
Bransford & Donovan, 2005; Corcoran, 2003; 
National Research Council, 1999; Wiske, 1998). 
Built into the design as well was prior knowledge 
about the principles of effective PD, as described 
above. Particularly important was research 
demonstrating the importance of using existing 
expertise in providing PD, gaining systemic sup- 
port, and giving teachers opportunities to actively 
build knowledge. While use of prior knowledge 
did not always translate into success, it provided a 
framework to focus the work on the improvement 
of teaching and learning. 

Using Prior Knowledge 

The LSC design was also characterized by 
mechanisms for using the knowledge of what 
works — particularly mechanisms for partnerships, 
changes in policy, and capacity building. LSCs 
involved major partners as stakeholders, includ- 
ing universities, research institutes, and business- 
es. The most beneficial type of partnership was 
collaboration with a powerful external partner that 
could provide expertise, resources, and a continu- 
ing presence for reform. As one K-8 science eval- 
uator said about an LSCs partnership with a uni- 
versity: 

An important outcome of the project has 
been the development of the "partnership" 
among the five districts and the university. 

. . . Each partner has brought considerable 
local strengths to the table to share with 
the others; each has been willing to 
explore new ideas presented and to reex- 
amine local practices that stood in the way 
of achieving their goals. the grant ends, 

there is every indication that this partner- 
ship will endure. (Banilower et ah, 2005, 
p.71) 

Partnerships could not accomplish LSC goals 
alone; they had to be accompanied by plans for 
institutionalizing policy change that could miti- 
gate factors in local contexts that threatened con- 
tinuation of the work. NSF required proposers to 
have a plan for developing a supportive context 
for reform. Policies supportive of projects were 
indeed put in place, providing guidance and 
incentives and drawing attention to targeted 
aspects of teaching and learning. Overall, LSC 
districts were much more likely to have support- 
ive policies in place in their last year than in their 
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first (Horizon Research, Inc., 2005). Formal adop- 
tion of the project instructional materials and 
aligned curricular frameworks were perhaps most 
critical to this accomplishment. 

Another mechanism of the design was capaci- 
ty-building strategies. Over time, both NSF and 
project leaders realized the need for stronger 
efforts to build capacity by developing a cadre of 
teacher leaders who had effectively used project 
materials and could encourage such use among 
other teachers. Projects that were able to prepare 
teacher leaders adequately left an important lega- 
cy in their districts. Also important to capacity 
building was gamering support from principals 
for the work of teacher leaders. An important les- 
son was that project designs needed strategies for 
building capacity within the system, not just 
imported expertise and resources. 

Evaluation 

As a further design feature, NSF recognized 
the importance of program evaluation that would 
provide information helpful for mid-course pro- 
ject adjustments and that could be aggregated 
across projects. Ongoing evaluation work proved 
helpful for building a shared understanding of the 
vision for instructional change. Classroom obser- 
vation protocols developed for the evaluation 
helped a variety of users to better understand the 
elements of high-quality teaching and to look 
beyond surface changes to examine the accuracy 
of classroom content and the extent of student 
conceptual learning. Savvy project staff were able 
to use data collection instruments and training 
materials to create change in PD providers, for 
instance helping teacher leaders understand the 
instructional vision and getting district leaders to 
focus on policy alignment. The LSC experience 
argues for using data gathered during a program to 
inform ongoing reform and to make sure that pro- 
grams have the capacity to disseminate such data 
effectively. Related to including evaluation in the 
design is establishing a community of learners 
among funders, project personnel, and evaluators. 
When means — formal meetings and informal net- 
works — exist for collaboration of project leaders, 
information needed to improve design can be 
shared and taken back to project sites. NSF 
encouraged the development of such communities 
of learners through PI meetings and an electronic 
communication system, and these improved both 
PD efforts and the evolution of NSF funding 
strategies, prescriptiveness, and evaluation. 



Time 

Finally, a specific feature of the LSC design 
was the expectation that reform would take time 
and require support for continued improvement. 
As the length of the grants, the expectation for 
large numbers of PD hours, the ongoing evalua- 
tions, and the encouragement of systemic support 
suggest, NSF expected that the work of the LSCs 
would take place over a long period and that to 
achieve impact, the enhancement efforts would 
have to work their way through the system. In this 
regard, the design was aligned with the consensus 
view of effective PD. 

The LSCs did indeed design for sustained 
reform, but were they effective? Evaluation 
results suggest that they were, with important lim- 
itations. 

Impact 

The LSC initiative shows that well-designed, 
systemwide PD can have significant impact on 
teachers, teaching practice, and possibly on stu- 
dents. Although the initiative did not reach its 
very ambitious goal of providing all targeted 
teachers with 130 hours of PD, in part because of 
teacher turnover, setting such a goal resulted in 
more “press” to go beyond volunteers, and the 
projects reached larger proportions of district 
mathematics/science teachers than did many pre- 
vious PD initiatives. The quality of PD was vari- 
able. While many sessions were excellent, evalu- 
ators also noted many missed opportunities for 
deepening teacher understanding of content and 
pedagogical strategies; in general, PD quality suf- 
fered to some degree from ineffective delivery by 
teacher leaders. Thus the quality across projects 
was not as high as the ambitious designs, based on 
the consensus view, intended. Nevertheless the 
initiative clearly enhanced teaching in a number 
of ways. 

Attitudes and Perceptions 

One finding of the LSC evaluation data was 
that teachers had more positive attitudes towards 
reform-oriented mathematics and science teach- 
ing. For various grade-level and subject groupings 
(K-8 science, K-8 mathematics, and 6-12 mathe- 
matics), teachers’ positive attitudes were correlat- 
ed with the extent of participation in LSC PD 
activities. Evaluators indicated that teachers left 
PD activities with greater enthusiasm for teach- 
ing, heightened awareness of how students learn, 
and willingness to modify teaching practices and 
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collaborate with colleagues. Teachers attributed 
new beliefs about how students learn — through 
questioning, discovery, and reflection — to their 
exposure to inquiry-based instruction in LSC pro- 
ject PD. Evaluators also noted changes in some 
teachers’ beliefs about the ability of all children to 
learn mathematics and science. 

Participation in LSC PD activities appears 
also to have had a positive impact on teachers’ 
perceptions of their pedagogical preparedness to 
teach mathematics and science. Across grade- 
level and subject groupings, their sense of pre- 
paredness, as measured by questionnaire data, 
increased with increasing participation in project 
PD. Even when extent of participation was con- 
trolled for, the sense of preparedness increased as 
the projects matured, indicating a systemic effect 
on preparedness. Results were similar, and some- 
what stronger, for teachers’ perceptions of their 
content preparedness. Again, more participation 
meant a greater sense of preparedness, with inter- 
view data showing teachers who had participated 
in 60 or more hours of PD more likely to report an 
impact on content knowledge than teachers with 
fewer hours. However, these results must be 
viewed with caution; evaluators noted that some 
teachers might have overrated their preparedness, 
and others might have felt less prepared as they 
grew to understand the complexities of effective 
instruction. 

Mathematics/Science Content of Lessons 

Teachers’ participation in LSC PD was linked 
to a number of positive outcomes in their instruc- 
tional practices. One key finding was that partici- 
pation was connected with higher ratings of the 
quality of mathematics and science content in 
observed lessons. Teachers with greater participa- 
tion were more likely to present significant and 
developmentally appropriate content and to pre- 
sent the subjects as a dynamic body of knowledge. 
Teachers grew in content knowledge over time 
participating in PD, as one evaluator of a K-8 
mathematics LSC explained: 

A deeper understanding of the content is 
reflected in the teacher-student interac- 
tions obserx’ed in classrooms. Teachers are 
better able to ask questions that have the 
potential for leading to greater conceptual 
understanding among the students. This 
would suggest greater understanding of 
the mathematics content. (Banilower et al., 
2005, p. 47) 



However, teachers’ capacity to address con- 
tent ranged widely, with evaluators citing evi- 
dence of teacher misconceptions and difficulty 
translating knowledge into student learning. 

Pedagogy of Lessons 

Another key finding was that participation in 
LSC PD was linked to increased use of reform- 
based pedagogy, including use of hands-on activ- 
ities, extended investigations, and work on mod- 
els/simulations. Over half of teachers participat- 
ing in the PD indicated that participation had 
influenced their selection of instructional strate- 
gies; those engaging in 60 or more hours of treat- 
ment were more likely to report this impact. 
Analyses of questionnaire data showed a positive, 
nonlinear relationship between PD participation 
and a composite measuring reported frequency of 
use of the investigative practices encouraged by 
the reform, with most of the impact occurring 
within the first 80 treatment hours, as shown in 
Ligure 1 . However, the increased confidence that 
teachers frequently reported in their ability to 
teach the subjects was sometimes misplaced, with 
observers noting lower levels of appropriate ped- 
agogy than self-reports would suggest. Such dis- 
crepancy between self-reports on preparedness 
and actual teaching behaviors may stem from 
teachers’ limited understanding of the depth of 
change that LSCs sought. 

The findings on pedagogy were thus decided- 
ly mixed. Observations early and late in the life of 
LSCs did support the notion that teachers’ use of 
instructional practices aligned with the LSC 
vision grew over time as a result of LSC PD, with 
teachers more often using investigative practices, 
questioning students effectively, and helping them 
make sense of lesson content. The extent of PD 
participation and use of LSC materials were both 
positively correlated with evaluators’ ratings of 
the quality of teachers’ questioning and sense- 
making strategies. The data also show, however, 
that fewer than half of lessons, even those of the 
teachers with the greatest PD participation, were 
rated highly in these two areas. Observations fre- 

1 

This composite (and others presented in this brief) was 
calculated as the percentage of total possible points on a 
group of questionnaire items related to a single construct. 

2 

The figures in this brief show "predicted" values of an 
outcome. These predicted values are calculated from the 
regression equation resulting from the analysis, which con- 
trolled for a number of teacher, school, and/or project char- 
acteristics. Thus these values represent the outcome for a 
“typical” LSC teacher at various levels of PD or a “typical” 
LSC project at various times from receiving funding. 
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Figure 1. Predicted Conposite Scores: Use of Investigative Practices 




quently revealed both rudimentary and accom- 
plished examples of mathematics and science 
instruction among LSC teachers. Many teachers 
were able to implement pieces of the reform, but 
few could deliver all the components to students. 
Much of the change in instruction cited by teach- 
ers stmck evaluators as superficial. Case studies 
revealed pedagogical weaknesses in LSC teach- 
ers, such as limited student engagement with con- 
cepts in modules, mechanical implementation of 
instructional materials, and infrequent use of 
sophisticated questioning and sense-making 
strategies. Overall, LSC efforts succeeded in 
shifting teachers’ thinking around pedagogy, but 
the program had more limited impact on teachers’ 
ability to fully operationalize their learning in this 
area. 

Materials Used in Lessons 

Also important was the finding that the use of 
district-designated instructional materials was 
positively correlated with more reform-based con- 
tent and pedagogy. In interviews and on question- 
naires, teachers who participated in a considerable 
amount of LSC PD reported more frequent use of 
the materials than did teachers with fewer hours. 
As with other outcomes, most of the change 
occurred in the first 80 hours of PD. Analysis of 
these data also showed a systemic impact, with 
teachers’ reported use of the designated materials 
increasing as projects matured, even after control- 
ling for teacher participation in the PD. 

While the use of instructional materials was 
fairly widespread in observed classrooms of LSC 
teachers, the quality of use varied tremendously. 



Evaluators were trained to use an observation pro- 
tocol that examined the design and implementa- 
tion of lessons, the learning climate of the class- 
room, and the quality of the mathematics/science 
content in the lesson. Observers used this infor- 
mation to rate the extent to which the lesson 
embodied the principles of teaching for under- 
standing (e.g., providing learning experiences 
aligned with developmentally appropriate learn- 
ing goals, using high-order questioning strategies, 
and providing students opportunities for sense- 
making). While some teachers did an excellent 
job of teaching for understanding, others conduct- 
ed lessons that focused on procedural knowledge 
rather than on the deeper conceptual knowledge 
that the materials were designed to foster. Evalua- 
tors reported that teachers in the early stages of 
learning were more likely to use materials 
mechanically, or to modify them inappropriately. 
Some teachers moved to activities on advanced 
concepts without an adequate foundation. Still 
others omitted activities, revised lessons, or added 
only loosely related supplementary materials, thus 
hampering the development of student conceptual 
knowledge. These adaptations often reflected 
teachers’ concerns about time constraints, class- 
room management, and/or high-stakes testing. 
Teachers’ lack of content knowledge also seemed 
to limit appropriate use of the materials; while 
many teachers engaged students with the materi- 
als, the lessons had little questioning and discus- 
sion to promote concept development. Some 
teachers came to perceive these shortcomings 
over time. As one K-8 science evaluator said: 
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One positive ejfect of the program is that 
teachers recognize the difference between 
hands-on activity in the science classroom 
and authentic, rigorous investigation. 
They realize that they are becoming more 
capable in the hands-on part of science 
exploration, but they also recognize that 
they are less fluent with the minds-on part 
of exploration. (Banilower et al, 2005, p. 

55) 

Evaluators did observe noticeable classroom 
improvements attributable to implementation of 
the materials. Overall, while use of the materials 
was far from perfect, evaluation data suggest that 
it enhanced the quality of instmction. 

Classroom Culture 

The evaluation revealed that ESC projects had 
an impact on the culture of mathematics and sci- 
ence classrooms. Analysis of teacher question- 
naire data found that time spent in ESC PD was 
associated with higher scores on a composite 
measure of practices that foster an investigative 
classroom culture, such as arranging classroom 
seating to facilitate student discussion, requiring 
students to supply evidence to support their 
claims, and encouraging students to consider 
alternative explanations. The typical participant 
scored 7.69 points higher (effect size 0.38 stan- 
dard deviations) than the typical nonparticipant, 
with most of the gain occurring in the first 80 
hours of PD. Evaluator ratings of observed 
lessons support this finding. Lessons were more 
likely to be rated highly for actively involving all 
students, engaging students intellectually, and cre- 
ating a climate of respect and rigor when taught 
by teachers with high levels of ESC PD. 

Nevertheless, evaluators also observed much 
classroom practice that remained far from inves- 
tigative. In one class, a teacher set the stage for 
inquiry by encouraging students to discuss what 
they already knew about the concept of melting. 
But when students showed misunderstanding of 
the concept, the teacher abandoned questioning 
strategies and resorted to merely telling students 
what melting was, thus relapsing into familiar 
teaching behaviors that precluded further investi- 
gation. Evaluators suggested that factors such as 
lack of content knowledge, lack of confidence 
with the investigative model and materials, and 
resistance to change contributed to such limita- 
tions on teachers’ capacity to create an investiga- 
tive classroom culture. 



Instructional Quality 

Evaluator ratings of instructional quality sup- 
port the findings on attitudes and practices. The 
extent of teacher participation in ESC PD and the 
use of the ESC materials were positively associat- 
ed with higher evaluator ratings of lesson quality, 
and the combination of PD and materials use had 
a greater effect than either alone. Lessons taught 
by teachers with at least 80 hours of PD and based 
on the materials were more than twice as likely to 
receive a high rating as lessons of untreated teach- 
ers not using the materials. An analysis of teacher 
observation data that accounted for the mediating 
effects of teacher attitudes, preparedness, and 
principal support found similar associations. 
Some differences were evident with lower levels 
of participation in ESC PD; lessons taught by 
teachers who had participated in at least 20 hours 
of ESC PD were more likely to be judged strong 
in a number of areas, such as the accuracy and sig- 
nificance of content, the climate of respect for stu- 
dent contributions, the engagement of students 
with important ideas, the likelihood of teachers’ 
questioning strategies enhancing conceptual 
understanding, the valuing of intellectual rigor 
and constmctive criticism, and the match between 
conceptual understanding and students’ develop- 
mental level. 

The ESC program also had an effect on the 
amount of time spent on science teaching in self- 
contained classes in the elementary grades. Eor 
example, Heck and Crawford (2004) found a pos- 
itive relationship between the extent of teachers’ 
participation in ESC PD and the minutes per week 
that elementary teachers spent on science instruc- 
tion (see Eigure 2). The effect size was moderate, 
equivalent to 6.21 minutes a week. Teacher par- 
ticipation in ESC PD fully explained this increase 
in instructional time as projects matured. There 
was no systemic effect for nonparticipating teach- 
ers in ESC districts. 

Student Impact 

Data on the direct impact of LSCs on students 
is quite limited at this time. While some projects 
included a design for assessing student impact, 
NSE did not require LSCs funded in the pro- 
gram’s first four years to examine this outcome. 
Of the 58 projects funded during this time, 16 pro- 
vided results from student impact studies. Later 
cohorts were required to study student impact, but 
many of these projects have not yet completed 
their studies. Reviews of the 16 studies, a Horizon 
Research, Inc. three-year, cross-site study of ESC 
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Figure 2. Time Devoted to Science Instructicn in Elementary 

Science Classes 




impact on elementary science achievement, 
teacher interview data, and principal question- 
naire data provide some information on student 
impact. The reviews (Banilower, 2000; Zhang & 
Wang, 2002) revealed evidence of positive impact 
on student achievement, attitudes, and course-tak- 
ing patterns in both mathematics and science 
LSCs. For example, in one LSC, Amaral, Garri- 
son, and Klentschy (2002) found a positive corre- 
lation between student achievement in both sci- 
ence and writing proficiency and the number of 
years a student received instruction from LSC- 
trained teachers using district-designated instruc- 
tional materials. However, constraints on research 
design in many of the studies, such as lack of a 
comparison group, made it difficult to judge 
whether the findings were attributable to the LSC. 

Data from the first two years of a cross-site 
science program study also showed some positive 
student impact. The study used a quasi-experi- 
mental, pre-test/post-test design to determine how 
teacher participation in LSC PD affected student 
achievement. First-year results indicated that the 
extent of teacher participation in LSC PD was a 
significant, positive predictor of student achieve- 
ment, after controlling for prior achievement and 
demographics. However, problems in data collec- 
tion prevented the analysis from including the 
amount of science instruction received or the 
extent of use of LSC instructional materials. 

Second-year results showed that teacher par- 
ticipation in LSC PD was a positive predictor of 
the amount of science instmction and the use of 



the instructional materials. Yet neither factor was 
a significant predictor of student achievement. 
Moreover, projects in the study were typically 
unable to require teachers to participate, so the 
possibility of selection bias was a concern. Final- 
ly, teacher interviews and principal questionnaires 
indicated a positive correlation between involve- 
ment in PD and reports of the impact of LSCs on 
student and school achievement. The available 
data thus suggest that the LSC program has had a 
positive impact on students. Forthcoming data 
from projects required to examine student impact 
should shed additional light on the extent of that 
impact. 

In summary, LSC PD had widespread impact 
on teaching and possibly on students, but the 
impact was not consistent and often not very deep. 
At the same time, challenges to consistent and 
deep implementation of investigative practices 
through LSC PD and materials were somewhat 
mitigated by an important adjunct to LSC core 
activities, the development of systemic support. 

Systemic Support 

NSF expected LSCs to build support through- 
out the system for comprehensive implementation 
of the PD design. Projects attempted to build sup- 
port in different ways. Important strategies were 
alignment of policies with the project vision, 
developing supportive structures and capacity for 
PD, and developing administrative support as well 
as external partnerships. Where system support 
was present, implementation of the LSC vision of 
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Figure 3. District Systems Coiposites 
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instruction seemed more effective and more like- 
ly to be sustained. 

Alignment 

The evaluation found that despite the fact that 
the policy context in which the LSCs operated 
was characterized by much administrative change 
and conflicting priorities, many LSCs were suc- 
cessful in aligning curriculum frameworks, 
assessments, and teacher evaluations with the 
LSC vision for mathematics and science instruc- 
tion, and in securing widespread adoption of 
exemplary materials by schools and districts. 
These successes fostered systemic changes in 
classroom practice, with both participating and 
nonparticipating teachers using practices and 
materials aligned with LSC principles. District 
leaders praised the alignment that LSCs promot- 
ed. One K-8 science administrator said that after 
the LSC work. 

Everyone understands the science agenda 
and is making sure schools are paying 
attention to it .. . We think of ourselves as 
a coordinated system, serving the schools 
. . . and that is a whole different way that 
this system has aligned itself (Banilower 
et ah, 2005, p. 73) 

The alignment of district policies and LSC 
principles took time; evaluators found that dis- 
tricts began with relatively weak systems for 
alignment with LSC reform, but alignment 
increased over the first two project years and was 
maintained to the end (see Figure 3). Riding the 
wave of standards-based reform in the 1990s, 



LSCs played critical roles in promoting both 
awareness and implementation of aligned policies 
throughout systems, particularly coordinated use 
of standards-based instruction and exemplary 
materials. Nonetheless, conflicting priorities 
sometimes interfered with alignment, which var- 
ied widely across projects. 

Structures 

Many LSC districts developed other stmctures 
that supported the LSC vision of PD systems for 
mathematics and science. Evaluators cited numer- 
ous ways in which LSCs influenced districts to 
support PD. In one urban district, more strategic 
planning and coordination across central 
office departments resulted in stronger sup- 
port for sustained PD for teachers and princi- 
pals. Some LSCs successfully pushed for manda- 
tory PD. Among the most important structures ini- 
tiated by LSCs were those to ensure PD for teach- 
ers new to the instructional materials. In one LSC, 
five districts jointly established a beginning 
teacher program to ensure training for newly hired 
teachers. Many systems had LSC-initiated PD 
structures, such as summer training, that remained 
in place after project funding ended. Evaluators 
deemed the most important legacy of the LSCs to 
be the expertise of teacher leaders. A number of 
LSC projects provided strong, ongoing support 
for teacher leaders, and continued to support these 
leaders in PD roles, such as mathematics coaches, 
after projects ended. However, LSCs sometimes 
underestimated what was needed to develop and 
sustain school-based PD, and overestimated the 
capacity of project staff and lead teachers to do 
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this work. Teacher leaders progressed at a rate 
slower than anticipated by LSC staff — evidence 
of a time lag between capacity-building efforts 
and teachers’ ability to make gains in competence. 
This finding shows again that effective PD needs 
support over time to succeed. 

Administrative Support 

Support from school and district administra- 
tors was crucial to the success of LSCs. LSC Pis 
typically cited principal support as the most 
important factor in determining teacher participa- 
tion in PD and in developing a supportive context 
for reform in schools. Both Pis and principals 
themselves noticed deeper commitment to PD as 
a result of LSC initiatives, and schools without 
principal turnover were significantly more likely 
to have positive outcomes on several measures of 
movement of LSC activities and materials into 
schools (Crawford & Banilower, 2004). On some 
measures, LSCs had diminished impact in schools 
with principal turnover. Overall, principal support 
was important but proved challenging for LSCs, 
with many principals paying lip service to reform 
without setting expectations for PD participation 
or materials use. 

On the other hand, efforts to gain the support 
of district administrators were less problematic. 
Where LSCs established strong working relation- 
ships with superintendents, school boards, cur- 
riculum directors, and others, the potential for 
sustained support increased significantly. In one 
multidistrict LSC, superintendents were engaged 
as co-PIs, and in turn the superintendents helped 
remove roadblocks to PD participation and mate- 
rials adoption. Across LSC projects, district lead- 
ers often demonstrated their support by allocating 
funds for replenishing science materials and pro- 
viding resources for training new teachers. Mid- 
level district administrators often proved the 
strongest and most stable allies; with low 
turnover, they could become a loyal group 
engaged in promoting the LSC vision and align- 
ing LSC reforms with district priorities. However, 
high superintendent turnover and competing pri- 
orities, such as the need to align materials with 
state assessments, did cause difficulty in sustain- 
ing district support. 

Partnerships 

As indicated earlier, LSCs were designed to 
involve external partners. The most effective 
external partners (e.g., universities, museums) 
provided not only resources but also help in estab- 
lishing stable structures for sustaining reform. 



such as centers for disseminating materials and 
PD. Access to expertise of university mathematics 
and science faculty also tended to be an advan- 
tage. Although attempts to build partnerships with 
universities and other assistance providers were 
limited by time, resources, and the demands of the 
LSCs’ primary mission of teacher enhancement, 
collaboration with partners sometimes resulted in 
support for LSC principles that lasted beyond the 
end of projects. 

Institutionalization 

LSCs aimed to institutionalize reforms 
beyond the end of NSF support. Given the rapid- 
ly changing education policy context, institution- 
alizing key elements of reform is essential for 
lasting impacts (Stone, Henig, Jones, & Pieran- 
nunzi, 2001). Through gamering both internal and 
external support, building district and school 
capacity, and encouraging adoption of supportive 
policies, LSCs took many steps to institutionalize 
a PD system aligned with the LSC vision for 
teacher enhancement. However, evaluators found 
that there was not a significantly greater likeli- 
hood of institutionalization of reform components 
at Year Five of projects than at Year Two (see 
Table 1). 

An erratic policy environment made institu- 
tionalization of PD activities and materials use 
difficult. Established support structures for PD 
were jeopardized by cuts in state and district fund- 
ing, changes in state assessments incompatible 
with the LSC vision, and particularly in science, 
moves to eliminate emphasis on PD in areas other 
than literacy and mathematics. Other barriers to 
institutionalization arose from LSCs themselves. 
For example. Pis often had more expertise for 
implementing PD activities than for dealing with 
issues related to policy and systems. Despite some 
success in institutionalization, such as the policy 
changes discussed earlier, in general LSC design- 
ers’ limited understanding of the scope of the 
work required for institutionalization limited their 
ability to translate LSC activities into systemic 
reform. Designers understood that institutional- 
ization was crucial to extending the scale of its PD 
efforts after funding ended, and they grasped that 
institutionalization relies on a major commitment 
of resources over time. Yet they were not able to 
grapple effectively with the challenges to this 
understanding that the reality of LSC implemen- 
tation posed. 

Many internal and external challenges affect- 
ed the success of LSCs. The most significant of 
these are described in the following section. 
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Table 1. Continuum Ratings of Likelihood of Institutionalization 

of LSC Reforms 

Percentage of Projects 



Year Two Final Year 

(N=58) (N=61) 

Level 1: Rubber Band Likely to Snap Back 0 0 

Level 2: LSC Reforms Likely to Gradually Fade Away 0 2 

Level 3: Minor Components Likely to B ecome Institutionalized 30 19 

Level 4: Components Likely to Become Institutionalized 63 62 

Level 5: Institutionalization of LSC Reforms Likely 8 17 



Challenges 

LSCs faced several sets of challenges that 
tended to limit the impact and staying power of 
reforms. A first set of cmcial challenges arose in 
the training of PD providers. While LSCs were 
typically rated highly in the quality of their prepa- 
ration of PD providers, evaluators also pointed to 
areas of weakness among providers that some- 
times limited impact. These weaknesses were 
seen in both content sessions (e.g., lack of rigor) 
and pedagogy sessions (e.g., lack of explicit dis- 
cussion of strategies). These limitations could be 
explained in part by the need for LSCs to press PD 
providers into service before they were fully pre- 
pared and in part by limited resources that con- 
strained the time and effort devoted to preparing 
providers. To address this challenge, more sus- 
tained effort around provider preparation and sup- 
port was needed, especially in areas that posed 
particular difficulties for teachers, such as unfa- 
miliar content, higher-order questioning and les- 
son closure. Projects might have trained providers 
better around the mathematics and science con- 
cepts underlying the student modules, might have 
illustrated student-centered instmction and stu- 
dent interaction more clearly, and might have 
more effectively presented ways to close lessons 
by helping students make conceptual connections. 

A related set of challenges for LSC PD was 
deepening teachers’ understanding of mathemat- 
ics and science content. In part, the emphasis that 
projects placed on materials and on the mechanics 
of the modules and kits distracted from broader 
content needs. In addition, some providers, partic- 
ularly teacher leaders, seemed underprepared for 
helping teachers make sense of content. More- 
over, PD sessions designed to deepen content 



knowledge and support teachers’ content needs 
during implementation sometimes failed because 
other more pressing teacher concerns, such as 
materials management, dominated the sessions. 
Again, these difficulties might have been 
addressed through better preparation and support 
for the providers. In addition, projects might have 
devoted a more equitable share of PD time to con- 
tent and have created more content-specific ses- 
sions matched to teachers’ needs. 

Further, supporting teachers during imple- 
mentation posed challenges for LSCs. While pro- 
jects typically provided teachers the supplies 
needed for the implementation of the modules and 
kits, they devoted far less attention to ongoing 
individual and small-group support, partly 
because of lack of resources and staffing. Teacher 
leaders, typically charged with providing ongoing 
support, were often spread thin and/or lacked the 
coaching and facilitation skills needed for engag- 
ing teachers effectively. Time constraints also led 
to difficulty in recruiting teachers for participation 
in the follow-up activities that were offered to 
build communities of learners and foster reflec- 
tion. Moreover, lack of school support and formal 
structures limited follow-up opportunities during 
the LSCs and pointed to difficulties in sustaining 
support for teachers beyond the grant. These chal- 
lenges suggest the need for cultivating greater 
awareness among district and school administra- 
tors, who could have encouraged teacher partici- 
pation in ongoing PD activities and fostered 
opportunities for collegial interaction and mutual 
support. 

Indeed, another set of challenges arose around 
engaging principals, who played key roles in 
determining the outcomes of LSCs — from 
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encouraging teachers to participate in LSC activi- 
ties, to enabling the work of teacher leaders, to 
making time for teachers to participate in ongoing 
site-based activities. Stable administrative leader- 
ship in schools was crucial, impacting the likeli- 
hood of teachers using LSC materials. LSCs 
struggled with balancing resources and effort 
devoted to teacher PD with reaching out to other 
stakeholders such as school leaders. Attention to 
principals was sometimes neglected altogether. 
Projects clearly had little control over some barri- 
ers affecting the extent of principal support, such 
as turnover, but they might have devised deliber- 
ate strategies for communicating their message to 
administrators through existing stmctures. Engag- 
ing principals in ongoing and fairly intensive 
ways would have been a worthy pursuit with 
potentially large payoffs both during and beyond 
the life of projects. 

Finally, projects found it challenging to reach 
all the targeted teachers. Despite efforts to pro- 
vide a range of opportunities, LSC projects were 
not able to engage most eligible teachers in 130 
hours of PD. Teacher turnover, state and district 
policies that reduced time available for project 
PD, and teacher reluctance to participate all con- 
tributed to this challenge. In some districts, the 
large influx of new teachers, as well as mobility 
across schools and subject areas, hindered efforts 
to meet the 130-hour goal. Many LSCs underesti- 
mated teacher turnover and were forced to impro- 
vise new strategies to meet the needs of new 
teachers. Projects might have better anticipated 
teacher turnover, making plans to include new 
teachers and to offer them incentives for partici- 
pation. Beyond this, projects could have looked 
for more effective strategies to make the case for 
reform with both teachers and administrators, and 
continued to make the PD opportunities attractive 
even to those resistant to the reform vision. 

Successes 

Along with these challenges, the LSC initia- 
tive experienced major successes, as has been 
shown. First, core evaluation data suggest that 
projects have done well in providing teachers with 
high-quality PD around mathematics and science, 
the initiative’s primary aim. Evaluators’ ratings 
for overall quality of PD programs were high 
across several dimensions. In particular, LSCs 
were strong in creating a culture conducive to 
teacher learning, and in preparing teachers to use 
high-quality materials and appropriate pedagogy 



in their classrooms. Further, ratings for overall 
program quality and the quality of individual ses- 
sions improved significantly over time, suggest- 
ing that LSCs became more adept at implement- 
ing high-quality PD as they matured and reinforc- 
ing the consensus view on PD that time matters. 
Indeed, time spent in PD seems an essential com- 
ponent for scale-up of instructional reform in 
mathematics and science. 

Success was also evident in the positive 
impact of LSC PD on teaching and learning. 
Teachers’ attitudes toward reform-oriented teach- 
ing and their perceptions of their content and ped- 
agogical preparedness improved with increased 
participation in LSC PD. Teachers’ participation 
in LSC PD was linked to the positive instruction- 
al outcomes described above, such as enhanced 
quality of lesson content and more frequent use of 
investigative practices and materials. Evaluation 
data also suggest that the quality of classroom 
instruction improves over time as teachers’ PD 
hours accrue, at least to the level of 80 hours. 
Although there were minor differences in impacts 
for mathematics and science teachers, overall, 
these results were equally evident in both sub- 
jects. 

Moreover, the widespread use of high-quality 
mathematics and science materials in the class- 
room by participating teachers points to program 
success. LSC PD focused heavily on giving teach- 
ers opportunities to explore the materials through 
practice, investigation, problem solving, and dis- 
cussion strategies that likely contributed to teach- 
ers’ willingness to use the materials with their stu- 
dents. Evaluation data suggest that the quality of 
instruction improved with teachers’ use of the des- 
ignated instructional materials, and that teachers 
were more likely to use the materials as they accu- 
mulated LSC PD hours. The LSC initiative also 
worked to ensure that the materials became estab- 
lished by promoting their formal adoption in par- 
ticipating districts. These efforts showed a sys- 
temic effect, increasing use over time among both 
participating and nonparticipating teachers in 
LSC districts. 

Finally, LSC projects had success in building 
stakeholder and policy support for the LSC vision 
and interventions. While LSCs began in an envi- 
ronment leaning toward reform, participating dis- 
tricts were less likely to have systems in place to 
support reforms before projects began. In con- 
trast, districts were much more likely to have 
these systems in place in projects’ last year, sug- 
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gesting a link between project efforts to build a 
supportive context and the institutionalization of 
PD systems, as well as the development of stake- 
holder and policy support. The adoption of LSC 
materials and aligned frameworks, district support 
of teacher leaders in varying capacities, and the 
development of external partnerships all demon- 
strate the success LSCs have had in building 
capacity to support and sustain reform. 

Conclusion 

The LSC initiative has shown that PD to 
enhance mathematics and science teaching can be 
implemented effectively at scale. When carefully 
designed and energetically supported, ambitious 
PD focused on instructional content and materials 
and sustained over time can change what happens 
in classrooms; impacts on teachers and their 
teaching were typically evident after approxi- 
mately 30 hours of PD, with further impacts 
detected through 80 hours of PD. The LSC evalu- 
ation suggests some implications of importance to 
those considering similar large-scale instructional 
reform through PD. First, if PD is going to have 
an impact, it needs to be focused on clear goals 
and delivered over time by well-trained providers. 
Competing and distracting PD efforts should be 
discouraged. Second, if PD is to move beyond 
business as usual, it must be based on content and 
practice and planned as a coherent set of strategies 
to develop teachers’ content and pedagogical 
knowledge. This work is difficult, but it does not 
require starting from scratch; most districts have 
internal capacity — such as teachers who can lead 
PD and principals who can align instmction with 
a reform vision — that can be used to move 
instructional change to scale. Third, alignment of 
district policies with instructional reforms and 
gamering the support of school/district adminis- 
trators is cmcial to the success and long-term sus- 
tainability of these reforms. Finally, stakeholders 
need to be aware that change takes time — and 
work to help teachers gain that time for learning. 
The findings from the LSC program thus support 
the consensus view about effective PD: the impor- 
tance of content-based PD, aligned with curricu- 
lum and assessment, focused on student learning, 
sustained over time, with collaboration among 
teachers, and administrative support. The LSC 
experience also points to some ways to make sure 
the common sense of that consensus becomes 
common in practice. 
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